Cryptographic protocols based on inner product spaces and group theory with a special focus on the use of Nielsen transformations

The topic of this thesis is established in the area of mathematical cryptology, more preciselyin group based cryptology. We give extensions of cryptographic protocols, develop new crypto-graphic protocols concerning the mathematical background and give modifications of them. Inaddition cryptographic analysis as well as examples are given. The focus lays on the developmentof new cryptographic protocols using non-commutative groups and of techniques, which are typ-ically studied in combinatorial group theory. Automorphisms on finitely generated free groupsare used, which can be generated by Nielsen transformations or Whitehead-Automorphisms.With the help of the Whitehead-Automorphisms we develop an approach for choosing automor-phisms randomly of the automorphism group Aut(F ), with F a finitely generated free group.Altogether twelve cryptographic protocols are explained. Among these are two extensions ofa (n, t)-secret sharing protocol, which is introduced by C. S. Chum, B. Fine, G. Rosenbergerand X. Zhang. Both extensions depend on the Closest Vector Theorem in a real inner productspace. The first one (Protocol 1) is a symmetric key cryptosystem and the second one is achallenge and response system (Protocol 2), which can be used by a variation as a two-wayauthentication. Furthermore, the HKKS-key exchange protocol by M. Habbeb, D. Kahrobaei,C. Koupparis and V. Shpilrain, which uses semidirect products of (semi)groups, is extended to anElGamal like public key cryptosystem (Protocol 3) and to a signature protocol (Protocol 4).There is an ongoing research about the HKKS-key exchange protocol with linear algebra at-tacks as well as research about suitable platforms, which also affects the ElGamal like publickey cryptosystem and the signature protocol. A short overview of the research is given in thisthesis.Furthermore, a purely combinatorial secret sharing scheme (Protocol 5) is introduced, whichuses a share distribution method explained by D. Panagopoulos for a (n, t)-secret sharing scheme.We show that this share distribution method is a special case of a multiple assignment schemeintroduced by M. Ito, A. Saito and T. Nishizeki. Furthermore, the introduced combinatorialsecret sharing protocol is shown to be similar to a variation of a secret sharing protocol ex-plained by J. Benaloh and J. Leichter. The idea of enhancing the combinatorial secret sharingscheme by using automorphisms on finitely generated free groups leads to two new secret sharingschemes. In addition a comparison to Shamir’s secret sharing scheme is given. The first one is asecret sharing scheme using a finitely generated abstract free group F , a finitely generated freesubgroup in SL(2,Q) and Nielsen transformations (Protocol 6). Protocol 6 is the basis forProtocol 7-12, which are also based on combinatorial group theory. The other secret sharingscheme (Protocol 7) uses a finitely generated free group F = 〈X | 〉, a Nielsen reduced setU 6= X and a Nielsen equivalent set V to U and gives therefore the final input for the newlydeveloped cryptographic Protocols 8-12, which are the main result in this thesis. Two newprivate key cryptosystems with similar modifications (Protocol 8 and Protocol 9) were de-veloped, another new private key cryptosystem (Protocol 10), a new ElGamal like public keycryptosystem (Protocol 11) and a new challenge and response system (Protocol 12), whichall use the combinatorial group theory and automorphisms on finitely generated free groups.Depending on the protocols the security is based on a linear congruence generator, the discretelogarithm problem in cyclic subgroups of the automorphism group of a finitely generated freegroup, the unknown algorithmic solution of the (constructive) membership problem in matrixgroups over the rational numbers or Hilbert’s Tenth Problem.